Iterative Projection Methods for Structured Sparsity Regularization
نویسندگان
چکیده
In this paper we propose a general framework to characterize and solve the optimization problems underlying a large class of sparsity based regularization algorithms. More precisely, we study the minimization of learning functionals that are sums of a differentiable data term and a convex non differentiable penalty. These latter penalties have recently become popular in machine learning since they allow to enforce various kinds of sparsity properties in the solution. Leveraging on the theory of Fenchel duality and subdifferential calculus, we derive explicit optimality conditions for the regularized solution and propose a general iterative projection algorithm whose convergence to the optimal solution can be proved. The generality of the framework is illustrated, considering several examples of regularization schemes, including `1 regularization (and several variants), multiple kernel learning and multi-task learning. Finally, some features of the proposed framework are empirically studied.
منابع مشابه
Solving Structured Sparsity Regularization with Proximal Methods
Proximal methods have recently been shown to provide effective optimization procedures to solve the variational problems defining the !1 regularization algorithms. The goal of the paper is twofold. First we discuss how proximal methods can be applied to solve a large class of machine learning algorithms which can be seen as extensions of !1 regularization, namely structured sparsity regularizat...
متن کاملAccelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints
Regularization of ill-posed linear inverse problems via l1 penalization has been proposed for cases where the solution is known to be (almost) sparse. One way to obtain the minimizer of such an l1 penalized functional is via an iterative soft-thresholding algorithm. We propose an alternative implementation to l1-constraints, using a gradient method, with projection on l1-balls. The correspondin...
متن کاملFast Iteratively Reweighted Least Squares Algorithms for Analysis-Based Sparsity Reconstruction
In this paper, we propose a novel algorithm for analysis-based sparsity reconstruction. It can solve the generalized problem by structured sparsity regularization with an orthogonal basis and total variation regularization. The proposed algorithm is based on the iterative reweighted least squares (IRLS) model, which is further accelerated by the preconditioned conjugate gradient method. The con...
متن کاملAlgebraic Iterative Reconstruction-Reprojection (AIRR) Method for High Performance Sparse-View CT Reconstruction
The reconstruction from sparseor few-view projections is one of important problems in computed tomography limited by the availability or feasibility of a large number of projections. Working with a small number of projections provides a lower radiation dose and a fast scan time, however an error associated with the sparse-view reconstruction increases significantly as the space sparsity increas...
متن کاملElectrical Impedance Tomography Image Reconstruction Using Iterative Lavrentiev and L-Curve-Based Regularization Algorithm
Electrical impedance tomography (EIT) is a technique for determining the electrical conductivity and permittivity distribution inside a medium from measurements made on its surface. The impedance distribution reconstruction in EIT is a nonlinear inverse problem that requires the use of a regularization method. The generalized Tikhonov regularization methods are often used in solving inverse pro...
متن کامل